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SENIOR  NONCOMMISSIONED  OFFICERS’  APR  OPINION  SURVEY 


I.  INTRODUCTION 

Inflation  of  ratingi  on  the  Airman  Performance  Report  (APR)  form  for  eenior  noncommissioned 
offloen  (NCO)  (E7  through  £9)  in  the  Air  Force  hit  eroded  the  usefulness  of  that  form  in  reoant  yean,  Ai 
early  ai  1971,  inflation  had  reached  an  aiymptoticilly  high  level  at  over  90  percent  of  all  tenter  NCOt  wen 
receiving  the  higheit  rating  pottible  for  overall  evaluation  of  job  performance  and  prom 'HI  on  potential.  The 
proble  m with  tuch  a large  degree  of  in  flation  it  that  the  homoge  neout  ratingi  mask  indivio  > • . dlffo  renoes  in 
performance  and  ability,  This,  in  turn,  mtket  differentiation  very  diffloult  for  those  required  to  make 
deoiiioni  with  retpeot  to  promotion,  atiignment,  and  other  penonnel  aotioni,  The  Air  Force  hat  become 
very  concerned  with  the  APR  inflation  problem  in  recent  yean,  and,  in  early  1975,  Air  Force  Military 
Penonnel  Center  (AFMPC)  wat  talked  with  developing  a new  tenior  NCO  APR  which  would  reduce 
inflation  of  ratings. 

Since  June  1973,  a number  of  itudiei  on  the  senior  NCO  APR  tyttem  have  been  conducted  by  the 
Alt  Force  Human  Resources  Laboratory.  Thete  have  included  an  analytls  of  rating  trenda,  a Q-aort  analysis 
of  the  procedures  of  the  FY  73  E9  selection  board,  questionnaires  sent  to  AFMPC  and  various  consolidated 
base  personnel  offices,  and  structured  Interviews  with  members  of  the  FY  75  B8  selection  board.  The 
purpose  of  the  present  research  effort  was  to  obtain  die  opinions  of  the  senior  NCOs  about  various  aspects 
of  the  current  APR  system  and  possible  changes  to  that  system,  using  a survey  (USAF  SCN  7642)  as  a basis 
of  analysis. 


II.  OBJECTIVES 

The  present  survey  had  three  specific  objectives.  One  objective  of  the  survey  was  to  have  the  senior 
NCO*  indicate  the  job  performance  factor*  which  they  felt  were  needed  to  evaluate  their  own  job 
performance  and  promotion  potential.  This  would  allow  those  most  familiar  with  a particular  job  to 
indicate  the  job  parameter*  that  need  to  be  measured.  Alio,  a comparison  of  response*  by  caieor  field, 
me) or  command  (MAJCOM),  time-in-grade,  and  other  variables  would  indicate  differences  within  the  senior 
NCO  foice  with  respect  to  which  factors  are  perceived  u being  most  important. 

Another  objective  of  the  survey  wat  to  determine  the  level  of  satisfaction  wfith  the  current  APR.  This 
is  important  information  for  two  reasons.  First,  it  gives  an  indication  of  whether  or  not  there  really  is  a 
problem  with  the  current  system  with  respect  to  user  satisfaction.  Also,  if  a new  APR  ayatem  is 
implemented,  it  provides  a baseline  with  which  to  compare  the  level  of  satisfaction  with  the  new  system  in 
the  fliture. 

In  the  past,  the  success  or  failure  of  evaluation  systems  in  the  Air  Force  hat  rested  largely  upon  user 
acceptance  and  satisfaction  with  the  system,  Therefore,  a third  objective  of  the  present  survey  was  to 
obtain  the  opinions  of  the  senior  NCOs  about  various  proposed  changes  to  the  APR  system.  This  would  give 
an  indication  of  the  probable  level  of  satisfaction  with  a particular  change  and  provide  guidance  on  APR 
development  before  going  through  the  costly  step  of  field  testing. 

In  addition  to  the  three  specific  objectives  of  the  survey,  there  was  an  overall  objective  on  a larger 
scale.  That  objective  was  to  allow  the  senior  NCOs  themselves  to  have  an  input  into  the  design  of  a 
penonnel  ayatem  that  has  such  a tremendous  effect  on  their  careers. 


III.  QUESTIONNAIRE  CHARACTERISTICS 

The  questionnaire  was  divided  into  four  sections,  The  first  section  (Appendix  A,  Part  I)  dealt  with 
demographic  and  penonal  Information  tuch  as  age,  grade,  sex,  Air  Foroe  specialty  code  (AFSC),  MAJCOM, 
etc.  The  last  three  sections  (Appendix  A,  Part  II,  III,  and  IV)  were  designed  to  correspond  to  the  three  main 
objectives  of  the  survey. 
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The  second  section  was  concerned  with  those  performance  factors  which  the  NCOs  felt  were 
nwemry  to  evaluate  them  in  their  own  particular  job.  Twenty-nine  factors  were  lilted,  and  the  NCOs  were 
asked  to  rate  each  factor  on  a five-point  scale  from  "always  necessary  for  adequate  evaluation  of  my  Job 
performance"  to  “never  necessary  for  adequate  evaluation  of  my  job  performance.”  The  29  factors  were 
compiled  from  those  appearing  most  frequently  in  performance  evaluation  literature,  assessment  center 
research,  and  past  APR  and  offioer  effectiveness  report  (OBR)  forms,  u well  as  those  mentioned  most  often 
In  Interviews  with  senior  NCOs  while  the  questionnaire  was  being  pretested.  In  addition  to  rating  each 
factor  on  a five-point  scale,  the  NCOs  also  indicated  in  a yes-or-no  fashion  whether  or  not  each  factor  was 
“needed  on  an  APR  to  measure  my  job  performance.”  The  reason  for  asking  the  second  question  waa  to 
decrease  some  of  the  overlap  that  may  have  occurred  in  simply  rating  each  factor  by  itself.  For  example, 
then  could  be  two  faotors  such  as  '’motivation”  and  “enthusiasm"  which  measure  the  same  underlying 
characteristic  of  job  performance.  While  both  factors  may  be  rated  as  being  necessary  for  evaluation  when 
viewed  in  Isolation,  only  one  needs  to  be  put  on  an  APR  form  since  they  both  measure  the  same 
characteristic. 

The  third  section  contained  1 S items  which  were  designed  to  determine  the  senior  NCOs’  opinions  of 
various  aspects  of  the  curnnt  APR  system. 

The  fourth  section,  which  consisted  of  10  items,  followed  up  by  asking  the  senior  NCOs  how 
acceptable  certain  changes  to  the  APR  system  would  be  to  them. 

The  last  page  of  the  questionnaire  was  left  blank  so  tha  NCOs  could  make  open-ended  comments 
about  the  APR  system  in  general  or  tha  questionnaire  in  specific. 


IV.  SAMPLE  CHARACTERISTICS 

The  questionnaire  was  mailed  to  10,000  Air  Force  NCOs  in  the  grades  of  E7,  E8,  and  E9,  The  sample 
was  atratified  by  2-diglt  AFSC  within  each  grade  with  oversampling  of  E8s  and  E9s  and  of  AFSCs  with 
amall  populations.  Oversampling  was  necessary  in  order  for  cell  frequencies  In  cross-tabulations  to  be  large 
enough  for  meanlngftil  analysis.  Overall,  questionnaires  were  mailed  to  5,000  E7s  (14%  of  total  E7 
population),  3,000  css  (32%  of  total  E8  population),  and  2,000  E9i  (44%  of  total  B9  population). 

Since  the  typical  return  rate  for  a survey  of  this  type  is  around  50  percent,  it  was  anticipated  that 
about  5,000  questionnaires  would  be  returned,  For  the  present  survey,  149  questionnaires  were  returned  as 
non-deliverable.  Of  those  that  were  delivered,  8,083  were  completed  and  returned  by  the  cutoff  date.  The 
resulting  return  rate  of  82  percent  wu  much  higher  than  was  expected. 

Table  1 shows  the  distribution  of  returns  by  grads,  race,  and  aex.  It  can  be  aeen  why  oversampling 
wu  neoeuary  in  some  instances.  For  example,  73  females  responded  to  the  questionnaire.  This  was  62 
percerft  of  the  female  population,  but  mads  up  only  one  percent  of  the  final  sample. 

Table  /.  Distribution  of  Returns  by  Grade,  Race,  and  Sex 


Varlahle 


•emsia 

% 


eoeuiitien 

% 


Grade  - 


Race- 


E7  3,836  47 

E8  2,450  30 

E9  1,791  22 

Black  701  9 

White  6,925  86 

Spanish  American  189  2 

American  Indian  39  1 

Aslan  American  35  1 

Other  174  2 


11 

26 

40 

15 

16 

• 

* 

57 


Guv 

Male 

7,993 

99 

17 

Female 

73 

1 

62 

V.  ANALYSIS 

The  analysis  of  the  lurvey  data  wu  Intended  to  provide  two  typei  of  information.  Pint,  an  overall 
feeling  for  the  opinion!  of  the  senior  NCOi  is  provided  by  a frequency  count  and  percent  of  the  number  of 
NCOs  who  choae  each  response  alternative  on  each  item  of  the  questionnaire.  Secondly,  it  was  necessary  to 
make  comparisons  among  NCOi  with  different  background  characteristics  to  determine  whether  or  not 
they  chose  significantly  different  response  alternatives.  Therefore,  the  frequency  counts  were  broken  out 
by  each  of  the  demographic  and  personal  information  variables  in  Part  I of  the  questionnaire.  A chi-square 
analysis  was  then  made  on  each  item  in  order  to  compare  the  response  trends  of  the  NCOe  in  the  various 
categories, 

In  addition  to  the  chi-square  analyses,  the  NCOs  were  compared  with  respect  to  the  way  they  rank 
ordered  the  job  performance  factors  In  Part  It  of  the  questionnaire.  The  set  of  factors  rated  on  a five-point 
scale  (Items  17  through  45)'  was  rank  ordered  according  to  the  total  number  of  rating  points  accumulated. 
The  other  set  of  factors  (items  46  through  74)  was  rank  ordered  according  to  the  number  of  times  each 
factor  had  been  chosen  as  being  necessary  on  the  APR  form.  Once  ranked,  the  two  sett  of  factors  were 
analyzed  using  the  Spearman  rank  correlation  coefficient  (r()  to  determine  the  degree  of  agreement 
between  the  rankings.  Also,  rankings  were  broken  out  by  each  of  the  background  variables  in  Part  I of  the 
questionnaire.  Because  more  than  two  sets  of  rankings  were  being  compared,  r,  was  an  Inappropriate 
statistic.  The  Kendall  coefficient  of  concordance  (W)  was  used  because  it  provides  a measure  of  the  degree 
of  agreement  among  a number  of  rankings. 


VL  RESULTS  AND  DISCUSSION 

'Table  1 shows  that  the  distribution  of  returns  conformed  very  closely  to  the  planned  sample.  Fifty 
percent  of  the  questionnaires  were  mailed  to  E7s,  and  47  percent  of  those  returned  were  from  E7s.  The  E8s 
comprised  30  percent  of  both  the  planned  sample  and  the  final  sample.  The  E9s  made  up  22  percent  of  the 
final  sample  and  20  percent  of  the  planned  sample. 

Part  11  • Rating  Factors 

Part  II  of  the  questionnaire  was  concerned  with  the  job  performance  factors  which  the  NCOs  rated  as 
being  needed  to  evaluate  their  job  performance.  The  average  rank  order  of  the  29  rating  factors  which 
appeared  in  Part  II  of  the  questionnaire  is  shown  in  Table  2.  The  rank  order  of  items  17  through  45  was 
averaged  with  the  rank  order  of  Items  46  through  74  to  arrive  at  the  overall  rank  order  seen  In  Table  2.  The 
correlation  between  the  two  sets  of  rankings  was  0.94.  This  high  degree  of  consistency  in  the  ranking  of 


libit  2.  Average  Rank  Order  of  Rating  Factors 


1.  Acceptance  of  Responsibility 

2.  Knowledge  of  Duties 

3.  Reliability 

4.  Leadership 

5.  Judgement 

6.  Supervisory  Capability 

7.  Quality  of  Work 

8.  Initiative 

9.  Bearing  and  Behavior 

10.  Oral  Communication 

11.  Planning  Ability 

12.  Decisiveness 

13.  Professional  Qualities 

14.  Utilization  of  Resources 

15.  Concern  for  Human  Relations 


16.  Motivation 

17.  Written  Communication 

18.  Emotional  Stability 

19.  Working  Relations 

20.  Executive  Ability 

21.  Flexibility 

22.  Adaptability  to  Strew 

23.  Listening  Skill 

24.  Equal  Opportunity 

25.  Ability  to  Train  Others 

26.  Learning  Ability 

27.  Quantity  of  Work 

28.  Self-Improvement  Efforts 

29.  Creativity 


factors  was  evident  no  matter  how  the  data  wen  broken  out.  Table  3 ihowi  the  Kendall  coefficient  of 
concordance  (W)  values  for  the  rank  ordering  of  factors  by  a number  of  background  variablei  including 
grade,  MAJCOM,  and  2-digit  AFSC.  At  can  be  teen  in  Tible  3,  all  W values  exceed  the  .001  level  of 
significance.  This  means,  for  example,  that  there  was  a significantly  high  degree  of  agreement  among  E7s, 
E8s,  and  E9s  with  nspect  to  the  way  the  fact  on  wen  rank  ordered.  This  high  degree  of  agreement  alto  held 
for  NCOe  in  the  various  MAJCOMs  and  caner  fields  as  well  at  the  other  variables  shown  in  Table  3. 


Table  3.  Kendal  Coefficient  of  Concordance 
Values  for  Rank  Ordering  of  Rating  Factors 


Variable 

KenSill  «W"  Value 

a 

Grade 

.95 

<.001 

Sex 

.99 

<.001 

Race 

.93 

<.001 

Years  in  Service 

.84 

<.001 

MAJCOM 

.93 

<.001 

2-Digit  AFSC 

.56 

<,001 

While  there  was  general  consistency  in  the  overall  rank  ordering  of  factors,  tome  differences  did 
appear  when  the  factors  wen  considered  individually.  In  Tablet  4,  S,  and  6,  factors  an  listed  which  had  a 
spread  of  at  least  six  positions  across  either  grade  (Table  4),  race  (Table  5),  or  2 -digit  AFSC  (Table  6).  In 
each  cate,  the  numbers  shown  Indicate  the  nlative  rank  order  of  the  factors  with  1 being  the  highest  ranked 
factor  and  29  being  the  lowest  ranked  factor.  For  example,  Table  4 shows  that  the  factor  “ability  to  train 
others”  decreased  in  importance  with  an  increase  In  grade.  E7s  ranked  it  as  the  20th  most  Important  factor 
while  E8t  ranked  it  at  24th  and  E9t  ranked  it  at  26th.  Factors  ranked  dlffenntially  by  race  an  teen  in 
Table  S.  Blacks  tended  to  rank  the  facton  “human  relations”  and  “equal  opportunity”  higher  than  any 
oth*r  radal/ethnlc  group.  Spanish  Americans,  on  the  other  hand,  ranked  “decisiveness”  higher,  and 
American  Indians  viewed  “oral  communication’’  as  being  mon  Important.  Asian  Americans  tended  to  rank 
"emotional  stability"  and  "executive  ability"  higher  than  the  other  groups. 

Faictors  listed  in  Table  6 are  those  which  were  ranked  differentially  by  ten  different  career  fields.  The 
differences  which  appear  in  Table  6 are  generally  what  would  be  expected.  The  factor  “bearing  and 
behavior"  was  ranked  highest  by  the  security  police  career  field  (81 XXX).  The  factor  "executive  ability" 
was  ranked  highest  by  the  administration  career  field  (70XXX)  and  lowest  by  the  aircraft  maintenance 
(43XXX)  and  transportation  (60XXX)  career  fields.  The  personnel  (73XXX)  and  supply  (64XXX)  career 
fields  tended  to  rank  "written  communication”  high  while  the  transportation  career  field  (60XXX)  ranked 
“utilization  of  resources"  higher  than  the  other  career  fields. 

The  differences  in  rankings  for  all  factors  listed  in  Tables  4,  5,  and  6 were  significant  at  the  .001  level. 
However,  some  of  the  differences  have  more  practical  significance  than  others.  For  example,  Table  4 shows 
that  while  the  factor  “ability  to  train  others"  was  more  important  to  E7s  than  to  K8s  and  E9s,  it  was  still 
ranked  only  20th  out  of  29  factors  by  E7s.  By  the  same  token,  the  factor  "knowledge  of  duties”  was 
ranked  among  the  top  ten  factors  by  all  grades  eve, n though  it  is  viewed  as  somewhat  more  Important  by  the 
lov'er  grades.  The  factor  “executive  ability,"  on  the  other  hand,  was  ranked  near  the  top  by  E9s(8th)  but 
near  the  bottom  by  E7s  (22d)  and  somewhere  near  the  middle  by  E8s  (16th).  Its  level  of  Importance, 
therefore,  shows  a substantial  Increase  with  an  increase  in  grade. 

Inspection  of  Tables  4,  5,  and  6 shows  that  few  factors  show  at  large  a shift  in  relative  position  at 
“executive  ability"  in  Table  4.  Mott  factors  listed,  even  though  their  rankings  were  significantly  different 
from  a probability  standpoint,  were  generally  ranked  within  ten  positions  of  one  another  by  all  of  the 
groups  being  compsred.  This  general  consistency  in  the  rankings  of  factors  by  all  groups  studied  wu  the 
predominant  result  from  Part  II  of  the  questionnaire. 
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Table  4.  Rating  Fact  on  Ranked  Differentially 
by  Grade 


Bin*  Olvsn  by 

IT  Kl  I* 

Ability  to  Train  Other* 

20 

24 

26 

Professional  Qualities 

10 

IS 

17 

Knowledge  of  Duties 

3 

6 

9 

Executive  Ability 

22 

16 

8 

Written  Communication 

17 

11 

11 

Table  3 . Factor*  Ranked  Differentially  by  Race 


Bank  llvan  by 

— “ 

Batin*  Pastor 

■lltll 

Spanish 

Amartaan 

Amartaan 

ineian 

Allan 

Amartun 

Whit* 

Betuing  and  Behavior 

11 

17 

13 

15 

9 

Human  Relations 

9 

13 

18 

18 

18 

Decisiveness 

17 

9 

14 

11 

12 

Emotional  Stability 

22 

20 

25 

15 

20 

Executive  Ability 

20 

16 

16 

10 

17 

Oral  Communication 

13 

14 

6 

16 

11 

Hanning  Ability 

14 

10 

11 

8 

10 

Equal  Opportunity 

12 

21 

24 

22 

24 

Utilization  of  Resources 

10 

12 

19 

13 

14 

Table  6.  Facton  Ranked  Differentially  by  2-Digkt  AFSC 


Ben*  liven  by 


Batin*  Pastor 

aexxx 

sexxx 

4SXXX 

eoxxx 

aexxx 

TOXXX 

7SXXX 

stxxx 

•oxxx 

Bearing  and  Behavior 

16 

12 

10 

13 

11 

12 

16 

9 

14 

Decisiveness 

9 

9 

12 

11 

15 

15 

13 

14 

13 

Executive  Ability 

11 

17 

21 

19 

17 

7 

11 

J7 

10 

Professional  Qualities 

15 

15 

13 

10 

14 

16 

15 

13 

17 

Initiative 

8 

8 

7 

14 

7 

8 

8 

10 

8 

Knowledge  of  Duties 

7 

S 

4 

4 

5 

9 

12 

7 

6 

Oral  Communication 

17 

10 

16 

12 

12 

11 

10 

15 

11 

Utilization  of  Resources 

13 

13 

11 

8 

13 

14 

14 

12 

15 

Written  Communication 

10 

11 

19 

16 

9 

10 

5 

11 

12 

Put  DI  • Opinion  of  Cumnt  APR 

Put  Ut  of  the  questionnaira  wu  timed  at  obtaining  the  opinions  of  the  senior  NCOs  toward  the 
current  APR  system.  When  asked  if  they  would  like  to  have  the  rating  factors  they  listed  at  being  important 
in  Part  □ of  the  questionnaire  replace  the  facton  found  on  the  current  APR  form  (item  76),  82  percent  of 
the  NCOe  agreed  that  they  would  like  to  have  the  questionnaire  factors  replace  the  cumnt  factors. 

In  response  to  a direct  question  about  how  satisfied  they  ue  with  the  cumnt  APR  system  (item  75), 
49  percent  of  the  NCOs  responded  that  they  were  In  all  ways  or  most  ways  satisfied.  Only  1 7 percent  were 
In  few  ways  or  no  ways  satisfied  while  34  percent  were  In  some  ways  satisfied.  This  pattern  of  responses 
was  consistent  both  across  grade  and  time-in-service  within  a particular  grade. 

In  probing  deeper  for  possible  reasons  for  dissatisfaction  with  the  current  APR  system,  the  NCOe 
were  asked  if  they  thought  their  careers  had  been  hurt  in  the  put  by  the  system  (item  85).  Overall,  34 
percent  of  the  NCOe  thought  their  careen  had  been  hurt  either  severely  or  moderately  by  the  APR  system. 
Another  20  peroent  felt  their  careen  had  been  hurt  slightly  while  36  percent  felt  it  had  not  been  hurt. 
Then  was  a definite  trend  by  grade  in  response  to  this  Item  u shown  in  Table  7.  While  43  percent  of  the 
B7s  indicated  that  their  careen  had  been  severely  or  moderately  hurt,  32  percent  of  the  E8s  and  only  19 
peroent  of  the  E9s  felt  the  same  way.  Since  E9s  have  been  u successful  with  the  system  as  possible  with 
respect  to  promotion,  it  is  not  suprising  that  they  wen  less  likely  to  feel  that  they  had  been  hurt  by  the 
system. 


Table  7.  Puling  that  Carur  Hu  Been  Hurt 
by  APR  vs.  Grade 


ansi 


My  cereer  Has  Seem 

ST 

% 

■a 

w 

at 

% 

Total 

% 

Severely  Hurt 

18 

11 

5 

13 

Moderately  Hurt 

25 

21 

14 

21 

Slightly  Hurt 

20 

20 

19 

20 

Not  Hurt 

28 

37 

54 

36 

Don’t  Know 

10 

11 

8 

10 

In  addition  to  the  trend  by  grade,  Table  8 shows  that  there  was  a trend  by  tlme-in-servicc  within  the 
grade  of  E7.  That  is,  the  feeling  that  one's  career  had  been  hurt  by  the  APR  system  tends  to  increase  u 
time-in-service  within  a particular  grade  inenases.  This  result  it  reuonable  since,  within  a particular  grade, 
the  longer  one  hu  bun  in  the  wrvice,  the  more  likely  it  is  that  he  or  she  hu  been  passed  over  for 
promotion. 


Tab  It  8.  Feeling  that  Carur  Hu  Been  Hurt  by  APR  ve,  Time 
In  Service  For  E7s 


My  Caraar  H«l  Seen 


Seventy 

Hurt 

Mederatvty 

Hurt 

>t 

H 

m 

Net 

Hurt 

Don't 

Knew 

Tia  (Vn) 

IS 

% 

% 

ta 

% 

Under  14 

6 

10 

13 

51 

19 

14-16 

4 

17 

20 

49 

11 

16-18 

9 

19 

23 

39 

10 

18-20 

16 

24 

23 

28 

10 

20-22 

21 

30 

18 

20 

10 

22-24 

27 

29 

18 

19 

7 

Over  24 

42 

28 

13 

12 

6 

10 

— ■■■Vi  ijy'Ai  ..wrim liaas^agg.  v.  AMMil 


The  nletlcnahip  between  not  end  btUtf  that  the  APR  ayitem  hat  hurt  one'i  career  appear*  In  Table 
9.  It  waa  noted  that  Hncki  were  moet  llkaty  to  balleva  that  their  careen  had  been  hurt  by  the  APRiyetem 
while  White*  wen  leaat  likely  to  feel  that  way. 

Table  9.  Feeling  that  Career  Hat  Been  Hun 
by  APR  vi,  Race 


My  Carwf  h«  Seem 

NMk 

% 

White 

OIMf 

« 

Severely  Hurt 

23 

12 

18 

Moderately  Hurt 

23 

21 

19 

Slightly  Hurt 

19 

20 

19 

Not  Hurt 

23 

38 

30 

Don't  Know 

11 

9 

14 

The  NCOe  wen  farther  aaked  If  they  felt  their  canen  will  be  hurt  in  the  fatun  if  the  current  APR 
eyitem  It  continued  (item  8*).  Table  10  ahowi  a tnnd  by  grade  even  more  prbnounced  than  that  teen  In 
retponte  to  item  85.  Since  the  APR  li  primarily  a promotion  tool,  it  waa  not  expected  that  E9t  would  think 
their  canen  would  be  hurt  In  the  fatun,  and  only  7 percent  naponded  that  it  would.  However,  45  percent 
of  the  E7i  felt  that  their  canen  would  be  hurt  to  eorne  degree  in  the  fatun  if  the  preient  APRiyitem  la 
continu'd.  The  tnnd  by  no*  teen  In  item  85  did  not  appear  In  item  86.  That  la,  Black*  wen  no  mom  likely 
than  Whitee  to  feel  their  canen  would  be  hurt  in  the  fatun. 

Table  10.  Feeling  that  Career  Will  be  Hurt 
by  APR  ve.  Grad* 


In  comparing  the  overall  reaponaet  to  itema  85  and  86,  it  appear!  that  then  waa  a more  podtive 
feeling  toward  the  fatun  of  the  APR  than  then  wai  with  the  paat.  Whereat  54  percent  of  the  NCOa 
thought  their  canen  had  been  hurt  to  tome  degne  in  the  put,  only  32  percent  thought  It  would  be  hurt  in 
the  fatun. 

Item  88  of  the  aueitlonnaln  aaked  the  NCOt  If  they  knew  other  aenior  NCOa  whoie  careen  had  been 
hurt  by  the  curnnt  APR  eyitem.  Only  5 percent  did  not  think  any  NCOi’  camera  had  been  hurt  while  63 
percent  thought  at  leaat  tome  NCOe'  canen  had  been  hurt.  However,  only  6 percent  thought  that  ail  or 
moet  of  the  aenior  NCO  force  had  been  hurt  cereer-wtae  by  tha  current  APR  ayttem, 

The  APR  la  the  primary  aource  of  Information  uaed  by  boarda  making  promotion  and  aaalgnment 
deddont,  Tha  primary  way  a peraoni  cureer  could  be  hurt  by  the  APR  ayttem  would  be  If  the  APR  were 
not  providing  the  promotion  and  aaalgnment  boarda  with  valid  Information  on  which  to  baa*  dedilona. 
Thanfote,  NCOa  wen  ukad  If  they  felt  that  valid  promotion  dedilone  could  bi  made  baaed  upon 


not  providing 


information  oontained  In  the  current  APR  (Item  83).  While  43  percent  of  the  NCOa  atrongly  or  moderately 
•greed  that  valid  promotion  dedilona  could  be  made,  48  percent  atrongly  or  moderately  diaigreed.  The 
trend  by  grade  In  naponae  to  this  queatlon  la  ihown  in  Table  11.  Aa  grade  Increued,  the  percentage  of 
NCOe  who  felt  that  the  promotion  ooardi  could  uie  the  APR  to  nuke  valid  dedilona  alio  tncreued.  It 


Table  11.  Feeling  that  Promotion  Board*  Cun 
Make  Valid  Deddoita  va.  Gnda 


Promotion  soirSi  Cm 
Moat  Voile  DteiMont 

% 

iittL 

as 

% 

ae 

w 

Total 

% 

Strongly  Agree 

4 

9 

9 

7 

Moderately  Agree 

30 

40 

43 

36 

Neither  Agree  Nor  Disagree 

10 

9 

9 

9 

Moderately  Diiagrie 

28 

25 

23 

26 

Strongly  Disagree 

27 

18 

15 

22 

makes  aenie  that  those  who  had  bean  promoted  by  the  system  would  be  more  likely  to  feel  that  the  system 
was  valid.  However,  even  among  the  E9s,  38  percent  moderately  or  strongly  disagreed  that  valid  promotion 
decisions  based  upon  the  APR  could  be  made.  This  increased  to  SS  percent  among  the  B7s. 

There  was  not  a strong  trend  by  time*in*aervlce  within  a particular  grade  in  response  to  item  83.  Diet 
is,  E7s  with  relatively  few  years  in  service  were  just  as  likely  to  think  that  promotion  boards  using  the  APR 
could  not  make  valid  decisions  as  E7s  with  more  years  in  service  even  though  the  latter  group  was  mote 
likely  to  have  been  passed  over  for  promotion. 

The  pattern  of  responses  to  item  84,  which  asked  if  the  NCOt  felt  that  valid  assignment/selection 
decisions  could  be  made  with  the  current  APR,  was  almost  identical  to  the  responses  to  item  83. 

Item  89  further  probed  the  NCOs’  perceptions  of  the  validity  of  the  promotion  boards.  The  NCOs 
were  asked  what  percentage  of  E7s  and  E8s  who  were  promoted  during  the  past  two  yean  deserved 
promotion  ahead  of  those  who  were  not  promoted.  The  responses  indicate  that  30  percent  of  the  NCOs  felt 
that  lets  than  20  percent  of  the  promotions  were  deserved  relative  to  those  who  wen  not  promoted.  Only 
17  percent  of  the  NCQi  felt  that  at  many  at  80  percent  of  Ote  promotions  were  deserved.  Table  12  shows 
only  a slight  trend  by  grade,  with  Ore  higher  grades  being  a little  more  positive  about  the  abilities  of  the 
boards  to  make  promotion  decisions. 


Table  12,  Percent  of  E7s  and  E8t  Who  Deserved 
Promotion  During  the  Past  Two  Yean  vs.  Grade 


a r«e« 

Portent  Who  OowrvoO 

a v 

it 

at 

Total 

Promotion 

% 

% 

% 

% 

0-20 

30 

29 

30 

30 

21-40 

18 

IS 

11 

16 

41-60 

22 

14 

14 

18 

61-80 

19 

21 

22 

20 

81-100 

12 

21 

23 

17 

It  should  be  noted  that  in  responding  to  item  89,  the  NCOt  were  comparing  promotion  selectees 
versus  non* selectees  by  grade.  However,  since  senior  NCOs  are  promoted  by  AFSC  rather  than  grade,  it 
could  be  that  at  least  a part  of  the  perceived  inequity  of  promotion  board  decisions  was  due  to  varying 
promotion  opportunities  in  diffennt  AFSCt.  For  example,  an  NCO  in  a very  competitive  AFSC  rmy  appear 
to  deserve  promotion  when  compared  to  all  NCOs  who  were  promoted,  but  not  when  compared  to  only 
those  NCOs  In  his  or  her  own  AFSC.  Even  to,  the  nsponses  to  items  83, 84,  and  89  Indicate  a widespread 
feeling  of  doubt  among  the  senior  NCOt  sampled  about  the  ability  of  promotion  and  selection/ MU gnment 
boards  to  make  valid  decisions  btsed  upon  the  information  available  to  them  in  the  current  APR.  This  is 
not  to  say  that  the  boards  do  not  make  valid  decisions,  but  that  the  perception  of  many  NCOs  it  that  they 
are  not  making  valid  decisions. 


J . . . i- — , ...ii  f.m 
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Since  inflation  of  ratings  on  die  APR  hu  become  a major  problem  for  promotion  and 
aelection/uiignment  boardi  trying  to  make  meaningful  differentiationi  between  NCOi,  the  eenlor  NCO» 
were  aeked  die  percentage  of  E7s,  E8»,  and  B9i  who  they  thought  were  given  maximum  ratings  on  the 
overall  evaluation  (election  of  the  APR  (item  SO). 

The  teiulte  (how  a general  undemanding  of  the  inflation  problem  at  79  percent  of  the  NCOl 
indicated  that  they  thought  oyer  80  peroent  were  receiving  maximum  rating!,  However,  it  li  turprising  that 
21  peroent  of  the  NCOi  did  not  teoognize  the  degree  to  which  inflation  had  taken  over, 

The  NCOi  ware  alio  aeked  what  the  primary  reaeon  wai  for  giving  maximum  ratlngi  when  they  wen 
not  deierved  (item  82).  Fifty  .seven  percent  atated  that  undeserved  maximum  ratinp  were  given  toeniun 
that  the  rater’s  people  get  promoted.  Only  two  percent  felt  that  everyone  who  received  a maximum  rating 
deaerved  it. 

Part  IV  • Changes  to  the  APR 

Part  IV  of  the  questionnaire  was  concerned  with  obtaining  the  opinions  of  the  senior  NCOi  about 
certain  possible  changes  to  the  APR  The  NCOi  expressed  overwhelming  support  for  having  two  separate 
APR  forms  for  evaluation  of  grades  B4-E6  and  B7-E9  (item  90).  However,  thoy  generally  disagreed  that 
different’ rating  factors  are  needed  to  evaluate  E7s,  B8s,  and  E9s  separately  from  one  another  (items  92  and 
93). 

Opinions  were  Just  about  evenly  divided  with  respect  to  whither  or  not  there  should  be  oareer  field 
specific  APR  forms  (item  91).  Overall,  SO  percent  of  the  NCOi  thought  that  all  career  fields  could  be 
evaluated  using  the  same  rating  factors,  while  46  percent  thought  that  separate  APR  forms  for  each  career 
field  were  needed  and  four  peroent  had  no  opinion.  There  was  a slight  trend  by  grade  in  response  to  this 
question  as  shown  in  Table  13.  As  grade  increases,  the  percentage  of  NCOs  who  believed  that  there  should 
be  career  field  specific  APRs  decreases.  This  is  reasonable  because  as  an  NCO  moves  up  in  grade,  duties 
usually  become  more  management  oriented  and  leu  career  field  specific, 


nblt  13.  Opinion  Toward  Career  Field  Specific 
APR  vs.  Grade 


' 

anlf. 

S7 

■e 

SI 

retd 

AM, 

w 

% 

% 

% 

No 

42 

52 

64 

50 

Yes 

53 

44 

34 

46 

No  Opinion 

5 

3 

2 

4 

Item  94  asked  the  NCOe  if  they  felt  that  one  set  of  rating  fketois  wu  needed  to  evaluate  present  job 
performance  and  a separate  set  of  factors  to  evaluate  promotion  potential  of  B7s  and  E8s.  Sixty-five 
percent  agreed  that  a separate  set  of  factors  wu  needed  while  24  percent  disagreed  and  10  percent  had  no 
opinion. 

Items  9S  through  97  concerned  the  poalbUity  of  having  a control  system  on  the  APRsimilar  to  the 
recently  implemented  OER  control  system.  First,  the  NCOi  were  uked  how  familiar  they  were  with  the 
OER  control  system  (item  95).  Overall,  61  percent  were  familiar  with  at  ieut  some  upects  of  the  system 
and  27  peroent  were  familiar  with  most  or  all  upects  of  the  system.  It  can  be  seen  from  Table  14  that  there 
wu  a slight  increau  in  understanding  of  the  system  with  an  increase  in  grade. 

Next,  the  NCOi  were  uked  if  they  agreed  that  the  overall  evaluation  portion  of  the  B7-E9  APR 
should  be  controlled  in  a manner  similar  to  that  of  the  OBR  (item  96),  Overall,  20  peroent  nelthei  agreed 
nor  diugreed.  Of  thou  who  had  an  opinion,  46  percent  were  in  favor  of  and  54  peroent  against  having  a 
control  system.  Thou  who  agreed  that  a control  system  would  be  desirable  tended  to  moderately  agree  (24 
percent)  rather  than  strongly  agree  (13  percent)  while  those  who  disagreed  tended  to  strongly  disagree  (29 
percent)  rather  than  moderately  disagree  (14  percent). 
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Table  14 . Familiarity  with  OBR  Control 
Syitam  vi.  Grada 


ante  . 

1 

■T 

it 

it 

Tetal 

Familiar  Wlthi 

* 

w 

% 

% 

All  Aspects 

3 

S 

9 

S 

Moat  Atpecta 

17 

24 

29 

22 

Some  Atpecta 

33 

36 

34 

34 

Few  Aapeota 

27 

25 

20 

25 

No  Aspects 

20 

11 

B 

14 

Table  IS  is  • eroartabulation  between  familiarity  with  the  OBR  control  lyitem  and  derin  for  an  APR 
control  system.  Generally,  for  thoio  familiar  with  at  least  tome  aspects  of  the  OBR  control  ayatem,  about 
half  wan  in  favor  of  and  half  againit  a control  ayatem  for  the  APR.  For  thoae  unfamiliar  with  the  OBR 
control  ayatem,  a majority  of  thoae  who  had  an  opinion  were  againit  having  a control  ayatem  on  the  APR. 
Table  IS  alao  ahowa  that  the  more  familiar  NCOa  were  with  the  OBR  control  ayatem,  the  more  likely  they 
were  to  have  an  opinion  about  a poaaible  APR  oontrol  ayatem.  Flfty*four  percent  of  thoae  lent  familiar 
with  the  OBR  control  wen  undecided  about  an  APR  control  while  that  waa  true  of  only  3 peroent  of  thon 
moat  familiar  with  the  OBR  control  ayatem.  Therofom,  while  lncnaaed  familiarity  meant  the  NCQi  were 
mon  likely  to  have  an  opinion,  the  opinloni  are  about  equally  divided  between  being  fbvorable  and 
unfavorable  toward  an  APR  oontrol  ayatem. 

Table  IS.  Opinion  Toward  Control  For  APR  va.  Familiarity 
with  OBR  Control 


FemlKirtty  wKH  0«W  Central 


Dttlr*  fw  Centre! 

All 

Aieeeti 

% 

Men 

"r* 

Stmt 

Aieeeti 

% 

Few 
Aiee  eii 
w 

Aieeeti 

% 

Strongly  Yet 

35 

21 

12 

8 

S 

Moderately  Yea 

16 

26 

31 

22 

10 

Undecided 

3 

4 

13 

27 

54 

Moderately  No 

6 

12 

IS 

17 

9 

Strongly  No 

41 

37 

29 

2S 

22 

Finally,  the  NCOa  were  aaked  if  they  thought  a control  ayatem  would  either  help  or  hurt  their  future 
caner  in  the  Air  Force  (ium  97).  Then  waa  a definite  trend  by  grade  in  reaponae  to  this  queition  ai  teen  in 
Table  16.  Since  the  APR  it  viewed  at  primarily  a promotion  Instrument,  80  peroent  of  the  B9a  felt  it  would 
have  no  effect  on  their  future  caner.  This  decrease*  to  43  percent  of  the  B8»  and  3S  percent  of  the  E7i. 
Overall,  48  peroent  of  the  NCOa  felt  that  controlling  the  overall  evaluation  lection  of  tne  APR  would  have 
no  effbet  on  their  ftiture  oaner.  Of  thoae  who  felt  It  would  have  an  affect,  tllghtly  mon  at  each  grade  level 
(bit  It  would  help  nthar  than  hurt  their  caner. 

Table  16.  Feeling  that  Control  Would  Help  or 
Hurt  Career  va.  Grade 


•race 

HewWewM  Centre* 

IT 

it 

i* 

Tetal 

Affect  Veur  eareerf 

% 

« 

% 

W 

Greatly  Help 

14 

12 

5 

11 

Moderately  Help 

25 

22 

6 

20 

No  Effect 

35 

43 

80 

48 

Moderately  Hurt 

16 

14 

5 

13 

Gnatly  Hurt 

10 

9 

4 

8 
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NCO  Comments 

In  addition  to  the  structured  Item*  in  the  quudonnaire,  the  lut  pip  wu  left  blank,  and  the  NCOe 
wen  instructed  to  uie  It  to  make  oommenti  about  the  eurvey  topic.  Approximately  3,000  of  the 
questionnaires  were  ntumed  with  commenti  Included.  About  half  of  the  comment!  noelved  were 
eubetantlve  and  could  be  quantified. 

The  most  frequently  occurring  comment  concerned  a Weighted  Airman  Promotion  Syetem 
(WAPS>type  promotion  eyitem  to  the  gradee  of  E8  and  E9.  Of  the  254  NCOe  who  mentioned  a WAPS-type 
promotion  eyitem,  96  peroent  were  in  favor  of  more  standardized  and  vtsible  promotion  prooedurea  while 
four  peroent  were  against  any  WAPS-type  eyitem  for  senior  NCOe, 

Another  frequently  occurring  comment  concerned  rating  officials  for  senior  NCOi.  The  oomment  was 
made  by  174  NCOe  tlmt  neither  junior  offlcen  nor  NCOa  of  the  same  grade  should  be  evaluating  senior 
NCOs.  Their  reasoning  was  that  junior  offlcen  do  not  have  the  experience  to  properly  evaluate  senior 
NCOs.  Also,  one  NCO  should  not  have  to  evaluate  another  NCO  with  whom  he  or  she  Is  competing  for 
promotion. 

A oomment  made  by  158  NCOs  was  that  less  emphasis  should  be  placed  on  evaluating  E9s.  A 
common  remark  was  that  B9a  should  be  given  either  abbreviated  APRs  or  only  referral  APRs  If  they  are  not 
doing  their  job  properly.  Most  felt  that  since  APRs  are  used  primarily  as  promotion  instruments,  It  Is  a 
waste  of  time  to  write  them  routinely  for  E9s. 

A number  of  comment!  were  received  against  a foroed  distribution.  Seventy  Kins  NCOs  commented 
that  a forced  distribution  would  be  too  political  and  that  those  with  the  most  visibility  would  get  the  best 
ratings.  A number  of  NCOs  (48)  wen  against  forcing  a quota  on  ratings  because  they  felt  that  some  units 
have  xll  outstanding  NCOs  while  other  units  have  a very  small  percentage  of  outstanding  NCOs.  A few 
comments  were  noelved  to  the  effect  that  a forced  distribution  would  cause  cutthroat  competition  in  the 
place  of  cooperation  and  unit  coheiiveneii. 

The  final  .ana  that  received  a substantial  number  of  oommenti  wu  the  narrative  section  or  word 
picture  of  the  APR.  Of  the  95  comments  that  were  received,  71  percent  felt  that  the  word  picture  should 
be  eliminated  while  10  percent  felt  the  amount  of  space  devoted  to  It  ahould  be  Increased.  Another  18 
percent  frit  that  the  word  picture  should  be  made  more  objective. 


Vll  SUMMARY  AND  CONCLUSIONS 

A questionnaire  wu  mailed  to  10,000  Air  Force  NCOs  In  the  grides  of  E7,  E8,  and  E9.  The  purpose 
of  the  questionnaire  wu  to  sample  the  opinions,  expertise,  and  attitudes  of  the  senior  NCOs  toward  various 
upects  of  perfonnance  evaluation  systems.  The  extremely  high  return  rate  of  82  percent  Is  an  Indication  of 
the  Interest  In  the  field  toward  this  topic. 

There  wu  general  agreement  smong  all  senior  NCOs  with  respect  to  which  rating  factors  are  needed 
to  evaluate  their  job  performance.  However,  the  factors  “executive  ability"  and  “written  communication" 
did  increase  In  Importance  with  an  lnorease  In  grade.  Therefore,  thou  factors  could  be  considered 
differentially  u an  indication  of  promotion  potential  for  E7s  and  E8s.  That  would  also  support  the  NCOs’ 
desire  for  separata  rating  factors  for  evaluation  of  job  performance  and  promotion  potential.  A substantial 
majority  of  NCOs  indicated  that  they  preferred  the  rating  factor*  they  indicated  u being  Important  on  the 
survey  to  thou  currently  found  on  the  E7-E9  APR. 

At  first  glance,  the  overall  level  of  utisfSctlon  with  the  current  APR  seemed  fairly  high  among  the 
NCOs.  About  half  were  definitely  satisfied  with  the  current  APR  while  about  one-third  were  fairly  neutrsl 
and  the  rest  were  relatively  dissatisfied.  However,  considering  the  fact  that  the  population  being  surveyed 
wu  thou  who  have  been  moet  successful  with  the  system,  the  level  of  satisfaction  wu  perhaps  lower  than 
might  have  been  anticipated. 

A slight  majority  of  NCOs  felt  their  career*  and  the  career*  of  at  leut  some  other  NCOs  have  been 
hurt  to  some  degree  by  the  current  APR  system.  However,  they  were  less  likely  to  fret  that  their  careers 
will  be  hurt  in  the  fbture  if  die  current  system  it  continued.  Almost  half  of  the  NCOs  did  not  feel  that  valid 
promotion  oi  assignment  decisions  can  be  made  based  upon  the  current  APR. 


IS 


With  r«tp«ct  to  poealble  change*  to  the  APR  eyatem,  the  NCOi  warn  in  flavor  of  maintaining  aaparata 
APR  forma  for  B4«B6i  and  E7*E9a.  Howavar,  thay  wara  about  avanly  divided  on  quaitiont  taking  about  the 
dedrablllty  of  carter  fltld  epedflo  APRa  and  having  an  OBR-type  control  ayttam  on  tha  overall  evaluation 
portion  of  the  APR. 

Several  auggaationt  were  made  by  tha  NCOi  on  tha  commenti  page  of  tha  queationnaire.  Many  NCOi 
axprattad  a dartre  for  more  atandardiaod  and  virtbla  promotion  prooadurat.  Thit  reaction  egainat  tha 
ourrant  promotion  prooadurat  oould  be  an  outgrowth  of  the  (baling  that  promotion  boarda  are  liot  able  to 
make  valid  dedtiont  bated  upon  information  contained  In  the  ourrant  APR  If  tha  APR  wara  improved  to 
tho  point  that  tha  NCOi  fait  that  valid  information  waa  reaching  tha  promotion  boarda.  than  pamapa  thay 
would  ba  mote  willing  to  accept  tha  deddona  of  tha  boarda  at  thay  are  preaently  operated. 

Another  area  of  concern  for  tha  aenior  NCOi  waa  with  their  rating  officiate.  Many  atated  that  junior 
offloert  do  not  have  tha  experience  nacetaaiy  to  evaluate  aenior  NCOe.  Many  alio  thought  that  an 
unhealthy  confllot  of  interact  ariaei  whan  a aenior  NCO  it  evaluated  by  another  aenior  NCO  of  the  tame 
grade  with  whom  he  or  aha  ia  competing  for  promotion. 

Finally,  many  aenior  NCOi  believed  that  too  much  empharta  ia  placed  on  evaluation  of  E9t.  They 
perceive  the  APR  at  primarily  a promotion  tool  and  think  that  too  much  time  ahould  not  be  apent 
evaluating  aomeone  who  cannot  be  promoted  anyway.  Suggeationa  were  made  to  theeffaot  that  evaluation 
of  E9t  could  be  de-emphaalzed  and  a nimpllfled  evaluation  form  auch  u that  uaed  for  civil  aervioe 
employee!  could  be  ueed  in  place  of  the  current  APR. 
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1.  What  ii  your  grada? 


J6CL- 

X»» 

A. 

MSgt 

3,836 

47 

B. 

SMSgt 

2,430 

30 

c. 

CMSgt 

1,791 

22 

2.  What  la  tha  praflx  of  your  duty  AFSC?  (If  your  AFSC  la  W31171,  tha 
praflx  la  W and  you  would  mark  "M"  on  your  anawar  ahaat.) 


N 

JL 

N 

JL 

Ji 

JL 

A. 

A 

407 

3 

p. 

L 

3 

0 

K. 

T 

372 

3 

B. 

D 

9 

0 

0. 

P 

13 

0 

L. 

U 

46 

1 

C. 

E 

29 

0 

H, 

Q 

2 

0 

M. 

W 

113 

1 

D. 

0 

22 

0 

I. 

R 

130 

2 

N. 

Othar 

30 

0 

E. 

H 

10 

0 

J. 

S 

39 

0 

0. 

I hava  no 
praflx 

6,473 

84 

* a 
a 

Anawar  quaatlona  3-7  regarding  your  duty  AFSC  ualng  tha  following 

a 

* 

raoponaaa  t 

a 

A.  0 

E.  4 1.  8 

a 

B.  1 

F.  3 J.  9 

a 

C.  2 

G.  6 

a 

a a 

D.  3 

H.  7 

a 

* 

a 

a 

a 

a 

a 


3,  What  la  tha  flrat  digit  of  your  duty  AFSC?  (If  your  AFSC  la  91471,  tha 
flrat  digit  la  9 and  you  would  nark  "J"  on  your  anawar  ahaat.) 


JL 

JL 

JL 

JL 

JL 

JL 

A. 

n 

Q 

E. 

1,338 

19 

I. 

273 

3 

B. 

349 

7 

F. 

700 

9 

J. 

617 

8 

C. 

997 

12 

G. 

976 

12 

D. 

1,232 

13 

H. 

1,136 

14 

What 

la  tha 

aacond  digit  of 

your  duty  AFSC? 

(If 

your  AFSC  la  91471,  tha 

aacond  digit  la  1 and  you  would  nark  "B"  on 

your 

anawar  ahaat.) 

JL 

JL 

N 

JL 

JL 

JL 

1,681 

21 

E. 

828 

10 

X. 

144 

2 

B, 

940 

12 

F. 

380 

3 

J. 

382 

7 

C. 

873 

11 

G. 

363 

3 

0. 

1,624 

20 

H. 

633 

8 

* N danotaa  tha  nuttbar  of  NCOa  who  choaa  aach  raaponaa, 
'*  ^ danotaa  tha  pareant  of  NCOa  who  choaa  aach  raaponaa. 


What  la  tha  third  digit  of  your  duty  A7SC?  (If  your  AFSC  la  91471,  tha 
third  digit  ia  4 and  you  would  mark  "E"  on  your  anawar  ahaat.) 


JL 

X 

JL 

X 

JL 

JL 

A. 

36S 

3 

E. 

713 

9 

X.  237 

3 

B. 

2,646 

33 

F. 

661 

8 

J.  31 

0 

C. 

1,893 

23 

G, 

430 

5 

D. 

843 

10. 

H. 

243 

3 

What  ia  tha  fourth  digit  of  your  duty  AFSC?  (If  your  AFSC  la  91471,  tha 
fifth  digit  la  7 and  you  would  mark  "H”  on  your  anawar  ahaat.) 


JL 

X 

JL. 

JL 

JL 

JL 

A« 

7 

0 

E. 

4 

0 

X.  24 

0 

B. 

7 

0 

F. 

50 

l 

J.  4,916 

61 

c. 

104 

1 

G. 

19 

0 

b. 

53 

1 

H.  2 

,871 

36 

What 

ia.  tha 

fifth  digit 

of  your  duty  AFSC? 

(If  your  AFSC  la  91471, 

tha 

fifth  digit 

la  1 and  you  would  mark 

"B"  cm 

your  anawar  ahaat.) 

JL 

JL 

JL 

JL 

JL 

JL 

A. 

4,442 

35 

E. 

488 

6 

I.  39 

i 

B. 

2,00V 

23 

F. 

81 

1 

J.  37 

0 

c. 

495 

6 

G. 

96 

1 

D. 

321 

4 

H. 

9 

0 

What 

la  tha 

auffix  of  your  duty  AFSC?  (If  your  AFSC  ia  91471B, 

tha 

auffix  la  B 

and  you  would  mark  "B" 

on  your 

anawar  ahaat,) 

JL 

X 

JL 

JL. 

N 

J 

A. 

A 148 

2 

G.  G 

33 

0 

M.  P 

8 

0 

B. 

B 60 

1 

H.  H 

1 

0 

N.  Q 

3 

0 

c. 

C 220 

3 

X.  J 

8 

0 

0.  S 

19 

0 

D. 

D 20 

0 

J.  K 

7 

0 

P.  T 

9 

0 

B. 

E 119 

1 

K.  L 

12 

0 

Q.  Othar 

35 

0 

F. 

F 29 

0 

L.  N 

1 

0 

R.  I hava 

no  7 

,268 

91 

auffix 
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What  waa  your  aga  on  your  laat  birthday? 


JL 

JL 

A. 

laaa  than  23  yaara 

16 

0 

B. 

26  but  laaa  than 

30 

31 

0 

C. 

31  but  laaa  than 

33 

808 

10 

D. 

36  but  laaa  than 

40 

3,040 

38 

B. 

41  but  laaa  than 

45 

3,076 

39 

F. 

46  but  laaa  than 

50 

854 

10 

0. 

over  30  yaara 

128 

2 

What  la  your  aax? 

i • 

JL 

JL 

A. 

Mala 

7,993 

99 

B. 

Faoala 

73 

1 

11.  Which  of  tha  following  do  you  conaidar  youraalf? 


A. 

Black 

JL 

701 

JL 

9 

B. 

Spaniah  or  Mexican  American 

189 

2 

C. 

American  Indian 

39 

0 

D. 

Aalan  American 

33 

0 

■E. 

Whlte/Caucaelan 

6,925 

86 

F. 

Other 

174 

2 

12.  What  la  your  Total  Active  Service  time? 


JL  JL 


A. 

Laaa  than  10  yaara 

17 

0 

B. 

10 

but 

leal  than 

12 

23 

0 

C. 

12 

but 

laaa  than 

14 

93 

1 

D. 

14 

but 

laaa  than 

16 

471 

6 

E. 

16 

but 

laaa  than 

18 

634 

8 

F. 

18 

but 

laaa  than 

20 

1,328 

16 

0. 

20 

but 

laaa  than 

22 

1,909 

24 

H. 

22 

but 

laaa  than 

24 

1,337 

17 

1. 

24 

but 

laaa  than 

26 

1,344 

20 

J, 

26 

but 

laaa  than 

28 

497 

6 

K. 

28 

or  more  yaara 

199 

2 

20 


To  what  major  coramand/organisation 

are  you  currently  assigned? 
N JL 

A. 

Aerospace  Defence  Command 

38  3 

5 

B. 

Air  Forca  Logistic*  Command 

119 

1 

c. 

Air  Forca  System*  Command 

362 

5 

D. 

Air  Forces  in  Europe 

657 

8 

G. 

Air  Training  Command 

771 

10 

F. 

Air  University 

53 

1 

a. 

Alaskan  Air  Command 

130 

2 

H. 

DoD  Agencies  (DNA,  DIA,  DC* 
DMA,  DIS,  DSA,  JCS,  OSD) 
Headquarters  Air  Force 
■Reserve 

61 

1 

i. 

2/ 

0 

J. 

Headquarters  Command,  USAF 

220 

3 

K. 

Headquarters  USAF 

85 

1 

L. 

Joint  Commands  (e.g,,  CINCEUR, 
CINCPAC,  CINSCO,  CINCNORAD) 

59 

1 

M. 

Military  Airlift  Command 

1,115 

14 

N, 

Pacific  Air  Forces 

474 

6 

0. 

Special  Operating  Agency 
(e.g.,  USAFA,  AFMPC,  ARPC, 
AFISC,  AFTEC,  AFDAA) 

211 

3 

P. 

Strategic  Air  Command 

1,605 

20 

Q. 

Tactical  Air  Command 

916 

11 

R. 

USAF  Security  Service 

213 

3 

s. 

Other 

600 

7 

14.  What  la  your  highest  level  of  education  now? 

N X 


A. 

No  high  school 

4 

0 

B. 

Some  high  school  or  GED  credits 

30 

0 

C. 

GED  certificate  or  diploma 

972 

12 

D. 

High  school  graduate 

3,071 

38 

E. 

One  or  two  years  of  college  or  vocational 
school  (include  Associate  degree) 

2,422 

30 

F. 

More  than  two  years  of  college 

1,006 

12 

0. 

Undergraduate  college  degree  (BA,  BS,  or 
equivalent) 

304 

4 

H. 

Graduate  Study  but  no  graduate  degree 

154 

2 

I. 

Graduate  College  Degree  (MA,  MS,  or  equivalent) 

83 

1 

J. 

Graduate  Study  beyond  Master's  degree  or  more 
than  one  Master's  degree 

14 

0 

K. 

Doctorate  degree  (PhD  or  equivalent) 

0 

0 

21 


15.  Whit  are  your  service  career  plane? 


N X 


A« 

Plan  to  stay  for  30  years  or  more, 
then  retire. 

1,911 

24 

B. 

Plan  to  stay  more  than  20  but  lass  than 
30  years,  than  retire. 

4,839 

60 

1 C* 

Plan  to  stay  20  years,  then  retire. 

724 

9 

Plan  to  stay  for  a while  but  probably 
not  until  retirement. 

16 

0 

| B. 

Plan  to  get  out  of  the  service  as  soon  as 
possible. 

100 

1 

F. 

Don't  know,  have  not  decided. 

463 

6 

16.  How  many  of  the  laat  10  yeare  have  you  spent  working  in  your  current 
career  field  (not  necessarily  at  your  prasant  skill  level)? 


A. 

Lesa  than  1 year 

JL 

180 

JL 

2 

B. 

1 but  less  than  2 

203 

3 

C. 

2 but  lass  than  3 

206 

3 

D. 

3 but  lass  than  5 

385 

5 

E. 

5 but  less  than  7 

431 

5 

F. 

7 but  less  than  10 

895 

11 

0. 

All  10 

5,765 

71 

22 
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On  the  previous  list  you  indicated  the  importance  of  each  rating  factor. 
However,  some  of  the  factors  which  you  indicated  to  be  important  may  be 
measuring  similar  qualities  so  that  all  of  them  may  not  be  needed  on  an  APR 
to  adequately  measure  your  job  performance.  Therefore,  for  the  list  below, 
please  indicate  only  those  factors  which  you  feel  should  be  Included  on  an 
APR  to  adequately  measure  your  own  job  performance.  Do  this  by  marking  in 
the  "A"  space  next  to  the  numbers  on  your  answer  sheet  corresponding  to  the 
factors  which  you  feel  are  necessary  to  adequately  evaluate  the  performance 
of  your  job.  You  may  pick  anywhere  from  one  to  all  twenty-eight  factors 
listed.  Mark  in  the  "B"  space  for  those  factors  which  you  feel  are  not 
needed  on  the  APR  form, 

A *>  needed  on  an  APR  to  measure  my  job  performance 
B ■ not  needed  on  an  APR  to  measure  my  lob  performance 


A B 


N 

% 

N 

H 

46. 

Acceptance  of  responsibility 

7,778 

96 

283 

4 

47. 

Adaptability  to  Btress 

4,116 

51 

3,903 

49 

48. 

Ability  to  train  others 

3,838 

48 

4,180 

52 

49. 

Bearing  and  behavior 

5,896 

73 

2,138 

27 

50. 

Concern  for  human  relations 

5,235 

65 

2,803 

35 

51. 

Creativity 

2,450 

31 

5,570 

69 

52. 

Decisiveness 

5,685 

71 

2,340 

29 

53. 

Emotional  stability 

4,500 

56 

3,533 

44 

54. 

Encouragement  of  good  working  relations 

4,771 

59 

3,263 

41 

55. 

Executive  ability 

5,099 

63 

2,933 

37 

56. 

Exhibits  professional  qualities 

5,586 

69 

2,459 

31 

57. 

Flexibility 

4,326 

54 

3,718 

46 

58. 

Initiative 

6,416 

80 

1,630 

20 

59. 

Judgement 

6,710 

83 

1,339 

17 

60. 

Knowledge  of  duties 

6,747 

84 

1,310 

16 

61. 

Leadership 

6,997 

87 

1,054 

13 

62. 

Learning  ability 

3,001 

37 

5,037 

63 

63. 

Listening  skill 

3,335 

41 

4,705 

59 

64. 

Motivation 

5,101 

63 

2,938 

37 

65. 

Oral  communication 

5,711 

71 

2,333 

29 

66. 

Planning  ability 

5,826 

72 

2,220 

28 

67. 

Promotion  of  equal  opportunity 

3,884 

46 

4,153 

52 

68. 

Quality  of  work 

6,662 

83 

1,395 

17 

69% 

Quantity  of  work 

3,196 

40 

4,839 

60 

70. 

Reliability 

6,729 

84 

1,330 

16 

71. 

Self-improvement  efforts 

3,251 

41 

4,774 

59 

72. 

Supervisory  capability 

7,036 

87 

1,025 

13 

73. 

Utilisation  of  resources 

5,630 

70 

2,415 

30 

74. 

Written  communication 

5,557 

69 

2,483 

31 

24 


PART  III 


I 


i 


*************************************** 

* Balov  you  will  find  a number  of  statements.  On  your  anawar  sheet,  naxt  * 

* to  tha  nunbar  of  aach  atatanant,  nark  In  the  apaca  with  tha  latter  that  * 

* corraaponda  moat  doaaly  to  tha  way  you  faal  about  the  atatanant,  Tha  * 

* APR  fora  and  ayatam  referred  to  aa  "currant"  la  tha  praaant  aat  of  * 

* document a and  procedures  for  evaluation  of  E7a , E8a,  and  B9s,  * 

*************************************** 

75,  To  what  extant  are  you  aatlaflad  with  the  currant  APR  ayatam? 


JL  JL 


Aa 

In  all  ways  satisfied 

320 

4 

B. 

In  moat  ways  satisfied 

3,621 

45 

c, 

In  some  ways  aatlaflad 

2,725 

34 

D. 

In  few  ways  satisfied 

1,003 

12 

E. 

In  no  waya  aatlaflad 

403 

5 

76*  1 would  Ilka  to  have  tha  rating  factora  that  I narked  aa  being  Important 

In  tha  previous  portion  of  this  survey  replace  tha  rating  factora  found 
on  tha  currant  E7-E9  APR  forn  (AF  Form  911), 


A. 

Strongly  agree 

JL 

3,171 

JL 

39 

B. 

Moderately  agree 

3,484 

43 

C. 

Neither  agree  nor  disagree 

1,154 

14 

D. 

Moderately  disagree 

145 

2 

E. 

Strongly  disagree 

104 

1 

77.  What  percentage  of  E7-E9a  do  you  think  are  given  maximum  ratings  for  all 
of  the  job  performance  factora  on  the  current  APR? 


N 

2 

A. 

0-202 

151 

2 

B. 

21-402 

151 

2 

C. 

41-602 

280 

3 

D. 

61-802 

1,490 

18 

% E. 

81-100* 

5,999 

74 

78*  What  percentage  of  E7-E9s  do  you  think  deserve  maximum  ratings  for  all 
of  the  job  performance  factors  on  the  current  APR? 


79.  If  ■on*  E7-E9e  art  givan  maximum  ratings  on  job  parformanca  factors 
whan  they  don’t  dssarve  them,  what  is  probably  the  primary  raaaon? 


JL 

JL 

A. 

Pressure  from  other  raters 

986 

12 

B. 

Desire  to  make  tha  parson  being  rated  "feel  good" 

661 

a 

C. 

Pressure  from  the  raters'  supervisors 

316 

4 

D. 

To  snsure  that  tha  people  gat  promoted 

To  justify  maximum  ratings  on  "overall  evaluation" 

4,186 

32 

E. 

761 

9 

F. 

Other  reasons 

1,021 

13 

G. 

I think  everyone  who  receives  maximum  ratings 
deserves  them 

130 

2 

80.  What  paroantaga  of  E7-E9s  do  you  think  ara  givan  maximum  ratings  on 


the 

overall 

evaluation 

N 

section  of  the  current  APR? 

JL 

A. 

0-20X 

118 

1 

B. 

21-40 X 

79 

1 

C. 

41-60% 

203 

3 

D. 

61-80X 

1,301 

16 

E. 

81-100X 

6,362 

79 

81.  What  paroantaga  of  E7-E9s  do  you  think  daaarva  maximum  ratings  on  tha 
ovarall  svaluation  taction  of  tha  currant  APR? 

N X 


A. 

0-2  OX 

1,362 

17 

B. 

21-40 X 

1,443 

18 

C. 

41-60% 

1,915 

24 

D, 

61-80 X 

2,337 

32 

E. 

81-100% 

781 

10 

82.  If  soma  E7-E9s  ara  givan  maximum  ratings  on  tha  overall  evaluation 
section  whan  thay  don't  daaarva  them,  what  it  probably  the  primary 
reason? 


A.  Pressure  from  other  raters 

B.  Pressure  from  tha  raters'  supervisors 

C.  Daalra  to  make  tha  person  being  rated  "feel  good" 

D.  To  ensure  that  tha  people  gat  promoted 

E.  Other  reasons 

F.  1 think  evtryone  who  receives  a maximum  rating 

deserves  It. 


N 

JL 

1,007 

12 

364 

5 

670 

8 

4,623 

57 

1,234 

16 

149 

2 

26 


% 


f- 


»■  ' 
jl 


t 


83.  I think  the  E8  and  E9  promotion  boards  can  maks  valid  promotion  decisions 
bassd  upon  information  contained  in  ths  currant  B7-E9  APR, 

N X 


A# 

Strongly  agree 

353 

7 

B. 

Moderately  agree 

2,919 

36 

C. 

Neither  agree  nor  disagree 

738 

9 

D. 

Moderately  disagree 

2,108 

26 

E. 

Strongly  disagree 

1,758 

22 

84,  1 think  valid  assignment  and  selection  decisions  can  be  made  based 

upon  information  contained  In  the  current  E7-E9  APR, 


A. 

8trongly  agree 

568 

7 

B. 

Moderately  agree 

3,233 

40 

C. 

Neither  agree  nor  disagree 

899 

11 

,D. 

Moderately  disagree 

2,022 

25 

E. 

Strongly  disagree 

1,330 

17 

83.  Do  you  think  your  cereer  has  been  hurt  at  some  time  in  the  past  ba cause 
of  the  present  APR  system? 


JL  JL 


A. 

Yes, 

it  has  been  severely  hurt 

1,069 

13 

B. 

Yes, 

It  has  been  moderately  hurt 

1,70B 

21 

C. 

Yes, 

it  has  been  slightly  hurt 

1,575 

20 

D. 

No, 

It  has  not  been  hurt 

2,936 

36 

E. 

I don't  know 

782 

10 

86.  Do  you  think  your  career  will  be  hurt  in  the  future  if  the  present  APR 
system  la  continued? 


JL  JL 


A. 

Yes,  it  will  probably  be  waver ely  hurt 

588 

7 

B. 

Yes,  it  will  probably  be  moderately  hurt 

1,071 

13 

C. 

Yes,  it  will  probably  be  slightly  hurt 

945 

12 

D. 

No,  it  will  probably  not  be  hurt 

4,337 

54, 

E. 

Z don't  know 

1,129 

14 

27 


How  many  APRs  presently  in  yo* „ omotion  folder  do  you  think  have  been 
in  the  past  or  will  be  in  the  future  harmful  to  your  chances  for  promotion? 


N 

% 

N 

% 

4,561 

56 

D. 

3-4 

288 

4 

1,760 

22 

E. 

5 or  more 

90 

1 

874 

11 

F. 

I don't  know 

495 

6 

Do  you  think  there  are  other  senior  NCOs  or  former  senior  NCOs  whose 
careers  have  been  hurt  to  some  degree  because  of  the  current  APR  system? 


Yes,  all  senior  NCOs  have  been  hurt 
Yes,  most  senior  NCOs  have  been  hurt 
Yes,  some  senior  NCOs  have  been  hurt 
Yes,  a few  senior  NCOs  have  been  hurt 
No,  no  senior  NCOs  have  been  hurt 


87 

395 

4,629 

2,574 

376 


Of  the  E7s  and  E8s  you  know  who  were  promoted  during  the  last  two 
years,  what  percentage  deserved  to  be  promoted  ahead  of  the  E7s  and 
E8s  you  know  who  were  not  promoted? 


0-20% 

2,375 

30 

21-40% 

1,255 

16 

41-60% 

1,413 

18 

61-80% 

1,625 

20 

80-100% 

1,372 

17 

PART  IV 

*************************************** 

* Below  are  a number  of  statements  about  possible  changes  in  the  E7-E9  * 

* APR.  Again  mark  'in  the  response  which  corresponds  most  closely  with  the  * 

* way  you  feel  about  each  statement.  * 

*************************************** 

90.  If  given  a choice  between  either  having  one  APR  form  for  evaluation  of 
all  NCOs  (E4-E9)  or  having  two  separate  forms,  one  for  E4-E6s  and  one 
for  E7-E9s  (as  is  currently  the  case),  I would  choose  to  have: 


A.  One  APR  form  for  evaluation  of  all  NCOs  469 

B.  Separate  forms  for  E4-E6s  and  E7-E9s  7,191 

C.  No  opinion/it  doesn't  matter  405 


91 


. If  given  a choice  between  either  evaluating  all  E7-E9  career  fields  using 
the  same  rating  factors  or  having  career  field  specific  APR  forms  so  that 
every  E7-E9  is  rated  only  on  factors  that  apply  to  his  or  her  specific 
career  field,  I would  choose  to: 

N _% 

A.  Evaluate  all  career  fields  using  the  same  rating  factors  4,043  50 

B.  Have  separate  APR  forms  for  each  career  field  3,712  46 

C.  No  opinion/it  doesn't  matter  308  4 

92.  In  my  career  field,  the  characteristics  which  make  a good  E7  are 

different  from  the  characteristics  which  make  a good  E8.  Therefore, 
different  rating  factors  are  needed  to  evaluate  E7s  and  E8s. 


N 


93. 


94. 


A.  Strongly  agree 

839 

10 

B.  Moderately  agree 

1,337 

17 

C.  Neither  agree  nor  disagree 

813 

10 

D.  Moderately  disagree 

1,577 

20 

E.  Strongly  disagree 

3,508 

43 

In  my  career  field,  the  characteristics  which  make 

a good  E8  are 

different  from  the  characteristics  which 

make  a good  E9.  Therefore, 

different  rating  factors  are  needed  to  evaluate  E8s 

and  E9s. 

N 

% 

• 

A.  Strongly  agree 

452 

6 

B.  Moderately  agree 

812 

10 

C.  Neither  agree  nor  disagree 

928 

11 

D.  Moderately  disagree 

1,693 

21 

E.  Strongly  disagree 

4,189 

52 

I think  the  APR  on  which  E7s  and 

E8s  are 

evaluated 

should  consist  of 

one  set  of  factors  for  evaluating  present  job  performance  and  a different 

set  of  factors  for  evaluating  potential  for  promotion  to  the  next  higtfer 

grade. 

* 

N 

% 

A.  Strongly  agree 

2,846 

35 

B.  Moderately  agree 

2,446 

30 

C.  Neither  agree  nor  disagree 

806 

10 

D.  Moderately  disagree 

723 

9 

E.  Strongly  disagree 

1,245 

15 

P.nnV 

29 


**************************************** 

* Under  the  new  officer  evaluation  system,  there  is  a limit  to  the  number  * 

* of  officers  who  can  receive  ratings  in  the  top  two  blocks  of  the  overall  * 

* evaluation  portion  of  the  OER  form.  That  is,  only  22%  of  the  officers  * 

* can  be  given  a rating  in  the  top  block,  and  a combined  total  of  only  50%  * 

* of  the  officers  can  be  given  ratings  in  the  top  two  blocks.  * 

**************************************** 

95.  How  familiar  are  you  with  the  new  OER  control  system? 

I 

N % 

A.  Familiar  with  all  aspects  of  the  system  389  5 

B.  Familiar  with  most  aspects  of  the  system  1,773  22 

C.  Familiar  with  some  aspects  of  the  system  2,738  34 

D.  Familiar  with  few  aspects  of  the  system  1,999  25 

E.  Familiar  with  no  aspects  of  the  system  1,170  14 

96.  I would  like  to  have  the  overall  evaluation  portion  of  the  E7-E9  APR 
controlled  in  a manner  similar  to  that  of  the  OER. 


N 

% 

A. 

Strongly  agree 

1,066 

13 

B. 

Moderately  agree 

1,926 

24 

C, 

Neither  agree  nor  disagree 

1,605 

20 

D. 

Moderately  disagree 

1,101 

14 

E. 

Strongly  disagree 

2,361 

29 

97.  To  what  extent,  if  any,  do  you  think  that  forcing  distribution  of  ratings 
on  the  overall  evaluation  portion  of  the  APR  would  either  help  or  hurt 
your  future  career  in  the  Air  Force? 

N % 


A. 

It 

would 

greatly  help  my  career 

883 

11 

B. 

It 

would 

moderately  help  my  career 

1,616 

20 

C. 

It 

would 

have  no  effect  on  my  career 

3,819 

48 

D. 

It 

would 

moderately  hurt  my  career 

1,038 

13 

E. 

It 

would 

greatly  hurt  ray  career 

680 

8 
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